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Abstract. Motivated by an original on-line page-ranking algorithm, starting 
from an arbitrary Markov chain (C n ) on a discrete state space 5, a Markov 
chain (Cn,M n ) on the product space <S 2 , the cat and mouse Markov chain, is 
constructed. The first coordinate of this Markov chain behaves like the original 
Markov chain and the second component changes only when both coordinates 
are equal. The asymptotic properties of this Markov chain are investigated. 
A representation of its invariant measure is in particular obtained. When 
the state space is infinite it is shown that this Markov chain is in fact null 
recurrent if the initial Markov chain (C n ) is positive recurrent and reversible. 
In this context, the scaling properties of the location of the second component, 
the mouse, are investigated in various situations: simple random walks in Z 
and Z 2 , reflected simple random walk in N and also in a continuous time 
setting. For several of these processes, a time scaling with rapid growth gives 
an interesting asymptotic behavior related to limit results for occupation times 
and rare events of Markov processes. 



The PageRank algorithm of Google, as designed by Brin and Page |9 a in 1998, 
describes the web as an oriented graph S whose nodes are the web pages and the 
html links between these web pages, the links of the graph. In this representation, 
the importance of a page is defined as its weight for the stationary distribution of 
the associated random walk on the graph. Several off-line algorithms can be used to 
estimate this equilibrium distribution on such a huge state space, the 'world largest 
Markov chain', they basically use numerical procedures (matrix- vector multiplica- 
tions) and refreshing mechanisms to switch from a set of nodes to another set of 
nodes. See Berkhin L 3j for example. 
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Although the off-line linear-algebraic techniques are currently dominating in 
PageRank computations, several on-line algorithms that update the ranking scores 
while exploring the graph have been proposed to solve some of the shortcomings of 
the off-line algorithms. The starting point of this paper is an algorithm designed 
by Abiteboul et al. pQ to compute the stationary distribution of a finite recurrent 
Markov chain. This algorithm has the advantage that it uses less computing and 
memory resources, in particular it does not require storing the link matrix as the 
off-line algorithms. Furthermore, the accuracy of its estimates is increasing with 
time. 

After the nth round of this algorithm, to each node x is associated a continuous 
variable W n (x), the "cash" at x. The initial state (Wo(x),x <E S) for the cash is 
some probability distribution on S. At each step of the algorithm, the page with 
the largest cash variable is visited. When a page x is visited its cash is distributed 
among its neighbors in the following way: a neighbor y receives p(x,y)W n (x) if 
P = (p{u, v)) is the transition matrix of the Markov chain. In that manner, the 
total amount of cash is invariant, i.e. (W n (x)) is a probability distribution on S for 
any n > 0. In the case of the web, p(x,y) = l/d x , if d x is the number of outgoing 
links from x. At the same time a variable h n (x) associated to x is increased by 
W n (x). The estimate of ir(x), the stationary distribution at x, is given by 

/. \ h n (x) 

K ' E ves h n (yy 

It is shown in Abiteboul et al. [T] that this quantity indeed converges to tt(x) as n 
gets large. 

A Markovian variant. Another possible strategy which chooses nodes at random 
to update the values of the cash variables is also considered in Abiteboul et al. Q] . 
Instead of considering the node with largest value of the cash, one considers a 
random walker who updates the values of the cash at the nodes of its random path 
in the graph. Note that both strategies have the advantage of simplifying the data 
structures necessary to manage the algorithm since there is no need of reordering 
the cash variables. It is shown in Litvak and Robert [21] that the corresponding 
quantity (fTJ also converges to the stationary distribution. The vector of cash at 
the nth visit is denoted by (V n (x), x € S). 

Since the total mass of cash is left invariant by these algorithms, it is assumed it 
is constant and equal to 1. The two sequences (V n (x),x € S) and (W n (x),x e S) 
are therefore Markov chains with values on probability distributions on the state 
space S. Outside the convergence of the quantity (TTJ to the stationary distribution, 
little is known on the asymptotic behavior of these Markov chain with continuous 
state space. 

Cat and Mouse Markov chain. It is shown, Theorem[TJ that the distribution of 
the vector (V n (x), x € S) can be expressed in terms of the conditional distributions 
of a Markov chain (C n ,M n ) on the discrete state space S 2 . The sequence (C n ), 
representing the location of the cat, is a Markov chain with transition matrix P — 
(j>(x,y)). The second coordinate, for the location of the mouse, (M„) has the 
following dynamic: 

- If M n ^ C„, then M n+1 = M n , 
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- If M n = C n , then, conditionally on M n , the random variable M n +i has 
distribution (p(M n ,y),y <E S) and is independent of C n +i- 

This can be summarized as follows: the cat moves according to the transition matrix 
P = (p(x, y)) and the mouse stays quiet unless the cat is at the same site in which 
case the mouse also moves independently according to P = (p(x,y)). 

The asymptotic properties of this interesting Markov chain are the subject of 
this paper. Intuitively, it is very likely that the mouse will spend most of the time 
at nodes which are unlikely for the cat. It is shown that this is indeed the case 
when the state space is finite and if the Markov chain (C„) is reversible but not in 
general. 

When the state space is infinite and if the Markov chain (C n ) is reversible, 
the Markov chain (C n ,M n ) is in fact null- recurrent. In this case, this implies 
that most of the time the mouse is "far away" from the favorite sites of (C n ). A 
precise description of the asymptotic behavior of the (non-Markovian) sequence 
(M n ) is done via a scaling in time and space for several classes of simple models. 
Interestingly, the scalings used are quite diverse as it will be seen. They are either 
related to asymptotics of rare events of ergodic Markov chains or to limiting results 
for occupation times of recurrent random walks. 

Outline of the Paper. Section[2]analyzes the recurrence properties of the Markov 
chain (C n ,M n ) when the Markov chain (C n ) is recurrent. It is shown that the 
distribution of the vector (V n (x) , x € S) is given by the distribution of the location 
of the mouse conditionally on the successive positions of the cat. A representation of 
the invariant measure of (C n , M n ) in terms of the reversed process of (C n ) is given. 
As a consequence, when the state space is infinite and (C n ) is ergodic and reversible, 
this measure has an infinite mass implying that (C n ,M n ) is null-recurrent. 

In Section [3J the cases of the symmetric simple random walks on Z d , with d = 1 
and 2, are investigated (the other simple random walks in higher dimensions are 
transient). Jumps occur at random among the 2 d neighbors. In the one-dimensional 
case, on the linear time scale t — > nt, as n gets large, the location of the mouse is of 
the order of \fn and the limiting process is a Brownian motion taken at the local 
time at of another independent Brownian motion. When d = 2, on the linear 
time scale t — ► nt, the location of the mouse is of the order of \/log n. In this case 
there is also a (weak) convergence of rescaled processes to a Brownian motion in 
R 2 on a time scale which is an independent discontinuous stochastic process with 
independent and non- homogeneous increments. 

For both cases, the main problem is to get a functional renewal theorem as- 
sociated to an i.i.d. sequence (T n ) of non-negative random variables such that 
E(Ti) = +oo. More precisely, if 

AT(t) = ^l {Tl+ ... +Ti < t} , 

i>l 

one has to find <f>(n) such that the sequence of processes (N (nt) / <j>(n) , t > 0) con- 
verges as n goes to infinity. When the tail distribution of T\ has a polynomial 
decay, several technical results are available. See Garsia and Lamperti [11] for ex- 
ample. This assumption is nevertheless not valid for the two-dimensional case. In 
any case, it turns out that the best way (especially for d = 2) to get such results 
is to formulate the problem in terms of occupation times of Markov processes for 
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which several limit theorems are available. This is the key of the results of this 
section. 

In Section 0] the case of the simple random walk in N with reflection at is 
analyzed. A jump of size +1 [resp. —1] occurs with probability p [resp. (1— p)] and 
the quantity p = p/(l — p) is assumed to be strictly less than 1 so that the Markov 
chain is ergodic. It is shown that if the location of the mouse is far away from 
the origin, i.e. Mq = n with n large, for the exponential time scale t — » p~ n t, the 
location of the mouse is of the order of n as long as t < W where W is a random 
variable related to an exponential functional of a Poisson process. See Bertoin 
and Yor [4]. After time t = W it is shown that the rescaled process {My tp - n ^/n) 
oscillates between and above 1/2 on every non-empty time interval. 

Section [5] introduces the equivalent of the cat and mouse Markov chain in the 
context of continuous time Markov chains. As an example, the case of the M/M/oo 
queue which can be seen as a discrete Ornstein-Uhlenbeck process is investigated. 
This stochastic process is known for its strong ergodic behavior: its invariant dis- 
tribution is sharply concentrated around (more than geometric decay) and also 
the fact that the time it takes to go to n from is of the same order as the time 
it takes to go to n from n — 1 (the last step is the most difficult). When Mq = n, 
contrary to the case of the reflected random walk, there does not seem to exist a 
time scale for which a non-trivial functional theorem holds for the corresponding 
rescaled process until the hitting time of 0. Instead, it is possible to describe the 
asymptotic behavior of the location of the mouse after the pth visit of the cat has a 
multiplicative representation of the form nF\Fi ■ ■ ■ F p where (F p ) are i.i.d. random 
variables on [0, 1]. 

It should be noted that despite the examples analyzed are specific, they are 
quite representative of the different situations for the dynamic of the mouse. For 
null recurrent homogeneous random walks, asymptotic results for the occupation 
times of Markov processes give the correct time and space scalings for the location 
of the mouse. When the initial Markov chain is ergodic and the mouse is away from 
the origin, the time scaling is given by the exponential time scale of the occurrence 
of rare events. The fact that for all the examples considered, jumps occur on the 
nearest neighbors, does not change this qualitative behavior. Under more general 
conditions analogous results should hold. Additionally, this simple setting has the 
advantage of providing explicit expressions for the constants involved (except for 
the random walk in Z 2 in fact). 



2. The Cat and Mouse Markov Chain 

In this section some properties of the process (V n (x),x S S) are investigated 
for a general transition matrix P = (p(x, y), x, y G S) on a discrete state space iS. 
Throughout the paper, it is assumed that P is aperiodic, irreducible without loops, 
i.e. p(x, x) = for all x € S and with an invariant measure n. Note that it is not 
assumed that tt has a finite mass. The sequence (C„) will denote a Markov chain 
with transition matrix P = (p(x,y)), it will represent the sequence of nodes which 
are sequentially updated by the random walker. 

The transition matrix of the reversed Markov chain (C*) is denoted by 

P*{x,y) = ^\p(y,x) 
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and, for y £ S, one defines 

H* = inf{ra > : C* = y} and H v = M{n > : C n — y}- 

A Representation of (V n (x),x £ S). The set of probability distributions on S is 
denoted by Vs- If Vq £ Vs, the total amount of cash initially is 1, then from the 
description of the algorithm it follows that for x £ S and t > 0, the relation 

(2) v n+1 {x) = Y,p{y,x)Vn{y)t {C n =y} + V n (x)l{C n7 ±x} 

y^x 

holds. Indeed, if a node y =/= x is visited then node x receives p(y,x)V n (y) of the 
cash from the node y. If the node x is visited then it distributes all its cash, and in 
this case, V n +i(x) is zero. The process (V n (x), x £ S) is a Markov chain with values 
in Vs the set of probability distributions on S. In the following it is shown that 
this continuous state space Markov chain can be expressed in term of a discrete 
state space Markov chain. 

For this purpose, a Markov chain (C„, M n ) on S x S referred to as the "cat and 
mouse Markov chain" is introduced. Its transition matrix Q — (q(-, ■)) is defined as 
follows, for x, y, z £ S, 

^ \q[(x,y),(z,y)} =p(x,z), Hx^y: 

=p(y,z)p(y,w). 

The process (C n ) [resp. (M n )] will be defined as the position of the cat [resp. the 
mouse]. Note that the position (C„) of the cat is indeed a Markov chain with 
transition matrix P — ■)). The position of the mouse (M n ) changes only when 
the cat is at the same position. In this case, starting from x £ S they both move 
independently according to the stochastic vector (p(x, •)). 

Cat and mouse problems are quite standard in game theory, the cat playing the 
role of the "adversary". See Coppersmith et at [10] and references therein. Here, 
of course, there is no question on the strategy of the mouse but only to analyze 
a Markovian description of the way the mouse may avoid, as much as possible, 
meetings with the cat. The evolution of the mouse (M n ) is analyzed in various 
settings in the rest of the paper. 

Let (T n ) denote the history of the motion of the cat, for n > 0, T n is the cr-field 
generated by the variables Co, Ci, . . . , C n . The cash process can then be described 
as follows. 

Theorem 1. If the Markov chain (C„,Af„) with transition matrix defined by Re- 
lation ([3]) is such that P(A/o = s|Co) = Vo(x) for x £ S then, for n > 0, the 
identity 

(4) {V n (x),x £ S) d = (P[M„ = x | T n -x],x £ S) 

holds in distribution. In particular, for x £ S, 

E(V n (x))=F(M n = x). 

Proof. For t > and x £ S, 
P(M„ +1 = x \T n ) = Y J HMn+i = x, M n = y\T n ) 

+ F(M n+i = x,M n = x\F n ). 
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For y ^ x, the Markov property of the sequence (C n , M n ) gives the relation 
P(M n+1 = x, M n =y\ T n ) = P(M n+ i = x\ M n = y,F n )x ¥(M n = y\F n ) 

= P(M n+1 = x \M n = y, C n ) x P(M n = y \ T n -i) 
= p(y, x) x P(M n = y | ^„_ 1 )l {Cn=2/} . 
In the case where the mouse does not move, 

P(M„ +1 =x,M n = x\ T n ) = ¥(M n = x | F n - X )l {Cn ±*\, 
by regrouping these terms, one gets the relation 

P{M n+1 = x | T n ) = ^P(M n = y | ^„_ 1 )l {Cn=2/} p(y, : E) 

+ P(M n = x | T n -i)l { c n *xy 

This shows that that the sequence (P[M n = x \ T n -i] : x £ S) satisfies Relation ©. 
The theorem is proved. □ 

Recurrence Properties of the Cat and Mouse Markov Chain. Since the 
transition matrix of (C n ) is assumed to be irreducible and aperiodic, it is not 
difficult to check that the Markov chain (C„, M n ) is aperiodic and visits with prob- 
ability 1 all the elements of the diagonal of S x S. In particular there is only one 
irreducible component. Note that (C n ,M n ) itself is not necessarily irreducible on 
S x S as the following example shows: Take S = {0,1,2,3} and the transition 
matrix p(0, 1) = p(2, 3) = p(3, 1) = 1 and p(l, 2) = 1/2 = p(l, 0), in this case the 
element (0,3) cannot be reached starting from (1,1). 

Theorem 2 (Recurrence). The Markov chain (C n ,M n ) on SxS with transition 
matrix Q defined by relation ^ is recurrent: the measure v defined as 




(5) v(x,y) =Tt(x)E x I 2^p(C*,y)j , x,y&S 

is invariant. Its marginal on the second coordinate is given by, for y G S 
V2{y) d = ^2 v{x, y) = E w (p(C , y)H y ), 

and it is equal to 7r on the diagonal, v{x,x) — n(x) for x € S. 

In particular, with probability 1, the elements of SxS for which v is non zero are 
visited infinitely often and v is, up to a multiplicative coefficient, the unique in- 
variant measure. The recurrence property is not surprising: the positive recurrence 
property of the Markov chain (C„) shows that cat and mouse meet infinitely often 
with probability one. The common location at these instants is a Markov chain 
with transition matrix P and therefore recurrent. Note that the total mass of v, 

u(S 2 ) = Y,^(p(C ,y)H y ), 

yes 

can be infinite when S is countable. See Kemeny et al. |18j for an introduction on 
recurrence properties of discrete countable Markov chains. 

The measure 1/2 on S is related to the location of the mouse under the invariant 
measure v. By Theorem [T] this quantity is closely related to the evolution of the 
cash process. 
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Proof. From the ergodicity of (C n ) it is clear that v{x, y) is finite for x, y € S. One 
has first to check that v satisfies the equations of invariant measure for the Markov 
chain (C n ,M n ), 

(6) v(x ) y)=^2u(z,y)p(z,x)+^2v(z,z)p(z,x)p(z,y), x,yeS. 

zyiy z 

For x, y € S, 

^v(z,y)p(z,x) 

= 5>(x)p*(x,z)E 2 (j2p(C*,y)\ =ir(x)E x ij2p{C* n ,y) 

z^y \n=l J \n=2 

and 

22v(z,z)p(z,x)p(z,y) = 22,ir{x)p*{x,z)p{z,y)E z ^ P( C n> 

zG5 z<£S V n=0 

The cycle formula for the invariant distribution of (C*) gives that 

E - ( E P^.*)! = E*TOE*(p(C5,*)) = = 1 

and, at the same time, the identity v(x, x) = tt(x) for x £ S. Hence, 

22 z)p{z, x)p(z, y) = ^2 n ( x )P* i x > z )p( z > v) = k{x)E x (p(C{ ,y)). 
zes zes 

The two last identities for v show that v is indeed an invariant distribution. 
The second marginal is given by, for y € S, 



^2^(x,y) = ^2J2tt(x)e x (p(C*,y)i{ H ,> 4 j 

xES t>l xeS 

= J2^(p(ct,y)i {H ,> t} ] 



t>i 



= XX (p(Co,y)l { H y >t } ) = E ff {p(C ,y)H v ) , 
t>i 

the theorem is proved. □ 

The representation {5| of the invariant measure can be obtained (formally) 
through an iteration of the equilibrium equations Since the first coordinate 
of (C„, M n ) is a Markov chain with transition matrix P and v is invariant measure 
for (C„, M„), the first marginal of v is thus equal to an for some a > 0, i.e. 

y) = a7r(a;), x E S. 



8 



N. LITVAK AND PH. ROBERT 



The constant a is in fact the total mass of v. In particular, from Equation |5[). one 
gets that the quantity 

M^) d = |X>(Q,y) ] , xes, 



def. 

yes 



is independent of x G S and equal to a. This can be proved more directly in the 
following way. One easily checks that x — * h(x) is harmonic with respect to (C*), 
i.e. that the relation 

h (x) = ^p*(x,y)h(y). 
yes 

holds for all x € S. One uses the classical result that since (C*) is a positive 
recurrent Markov chain, all harmonic functions are constant. See Neveu [22] . In 
particular, one has 

xeS yes \n=i J y es 

with the same calculation as in the proof of the above theorem. Note that the 
parameter a can be infinite. 

Proposition 1 (Location of the mouse in the reversible case). If (C n ) is a reversible 
Markov chain, with the definitions of the above theorem, for y £ S, the relation 

vi(y) = 1 - n(y), 

holds. If the state space S is countable, the Markov chain (C n ,M n ) is then null- 
recurrent. 

Proof. For y £ S, by reversibility, 
v 2 (y) =E„ (p(C ,y) H y ) =J2<x)p(x,y)^AH y ) 

X 

= J2<y)p(y> x ) E *( H v) = *(y) E y(H y - 1) = 1 - Av)- 

X 

The proposition is proved. □ 

Corollary 1 (Finite State Space). If the state space S is finite with cardinality N , 
then (C n ,M n ) converges in distribution to (Coo, Moo) such that 

(7) P(Coo = x, Moo — y) = cn(x)E x \ J2p(C*,y) j ,x,yeS, 

with 

c=l /j2^(p(Co,y)H y ) , 
/ yes 

in particular P(Coo = Moo — x) = cir(x). If the Markov chain (C n ) is reversible, 
then 

1 - w(y) 



P(Moo = y) 



N - 1 
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Tetali [28] showed, via linear algebra, that if (C n ) is a general recurrent Markov 
chain, then 

(8) ^E,(p(C ,z)ff,)<iV-l. 

See also Aldous and Fill [2]. It follows that the value c = l/(N — 1) obtained 
for reversible chains, is the minimal possible value of c. The constant c is the 
probability that cat and mouse are at the same location. 

In the reversible case, one gets the intuitive fact that the less likely a site is 
for the cat, the most likely it is for the mouse. This is however false in general. 
Consider a Markov chain whose state space S consists of r cycles with respective 
sizes mi, ■ ■ ■ , rn r with one common node 

T" 

5 = {0}U \J{{k,i) :l<i<m k }, 

k=l 

and with the following transitions: for 1 < k < r and 2 < i < nik, 

p{{k, i), (fc, i - 1)) = 1, p((k, 1), 0) = 1 and p(0, (fc, m k )) = -. 

r 

Define m = mi + TO2 + • • • + m r . It is easy to see that 

tt(0) = — ^— and n(y) = yeS- {0}. 

m + r m + r 

One gets that for the location of the mouse, for y 6 S, 

Tr(y)(m — nik + r), if y = (k, m^), 1 < k < r 
n(y), otherwise. 



u 2 (y) = E n (p(C ,y)H y ) = 



Observe that for any y distinct from and (k, rrifc), we have 7r(0) > n(y) and 
^2(0) > ^2(2/)) the probability to find a mouse in is larger than in y. Note that in 
this example one easily obtains c = 1/r. 

3. Random Walks in Z and Z 2 

In this section, the asymptotic behavior of the mouse when the cat follows a 
recurrent random walk in Z and Z 2 is analyzed. The jumps of the cat are uniformly 
distributed on the neighbors of the current location. 

3.1. One-Dimensional Random Walk. The transition matrix P of this random 
walk is given by 

p(x, x + 1) = — = p(x, x — 1), ieZ 

To get the main limiting result of the one-dimensional random walk, Theorem [3l 
two technical lemmas are first established. 

Lemma 1. For any x, e > and K > . 



Inn 1\ inf _ ^ (l+T lti )<K 

i—k 



1 

>w+oo \a<k<\x^n\ \fn 



where (Ti^) are i.i.d. random variables with the same distribution as the first hitting 
time of of (C n ), T x = inf{n > : C n = 0} with C = 1. 
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Proof. If E is an exponential random variable with parameter 1 independent of the 
sequence (Ti,,), by using the fact that, for u <G (0, 1), E(u Tl ) = (1 — y/1 — u 2 )/u, 
then for n > 2, 

/ 1 L*V»J \ 

logP I — ^ (1 + ri lfc ) < E I = LeV^J log (l - VI -e- 2 /v^J < 

Denote by m n the infimum of the assertion, the above relation gives directly 

P(m» < £7) < La;\/^Je" £ ^™, 

hence 

^ P (m» < E) < 

n=2 

consequently, with probability 1 there exists Nq such that, for any n > Nq, we have 
m n > £7. Since P(_E > if) > 0, the lemma is proved. □ 

Lemma 2. Let, form > 1, (T2.1) i.i'.d. random variables with the same distribution 
as T2 = inf{fc > : Cfc = 0} with Co = 2 and 

+00 

U„ = Xl 1 {ELi(2+T 2 , fc )<«}' 

£/ien i/ie process (u\ tn \/ ^fn\ converges in distribution to (£b(4)/2), where Ls{t) is 
the local time process at time t > 0/ a standard Brownian motion. 

Proof. The variable Ti can be written as a sum Ti + T{ of independent random 
variables T\ and T{ having the same distribution as T\ defined in the above lemma. 
For k > 1, the variable Ti t k can written as T\^k + Ti,2fc+i- Clearly 
^ +00 _^ ^ +00 

2 E 1 {ELi(i+Ti, fc )<"} " 2 " " 2 E 1 {ELi(i+T 1 , fc )<«}- 
Furthermore 

/ +00 \ / n— 1 \ 



E 1 {i:! =1 (i+T 1 ,,)<n}> ri ^ 1 = ( r «) =" E 1 



= n\, n > 1 



{C f =0}, 

=1 

where (C n ) is the symmetric simple random walk. 

A classical result by Knight [20] , see also Borodin [7 j and Perkins [24] , gives that 
the process {r\ n t\/y/n) converges in distribution to (Lg(i)) as n gets large. The 
lemma is proved. □ 

The main result of this section can now be stated. 

Theorem 3 (Scaling of the Location of the Mouse). If (Co, Mo) £ N 2 , the con- 
vergence in distribution 

lirn^ (^l=M [nti ,t> =*' (b^Lb,®)^ > 0) 



n — >+oo 

holds, where (Bi(t)) and (B%(t)) are independent standard Brownian motions on . 
and (LiB 2 (t)) is the local time process of (B^it)) at 0. 
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The location of the mouse at time T is therefore of the order of \/T as T gets large. 
The limiting process can be expressed as a Brownian motion slowed down by the 
process of the local time at of an independent Brownian motion. The quantity 
Lb 2 (T) can be interpreted, as the scaled duration of time the cat and the mouse 
spend together. 

Proof. A coupling argument is used. Take 

- i.i.d. geometric random variables (Gi) such that P(Gi>p) = l/2 p_1 forp > 1; 

- (C k ) and (Cj k ), j > 1, i.i.d. independent symmetric random walks starting 
from 0; 

and assume that all these random variables are independent. One denotes, for 
m = 1, 2 and i > 1, 

= inf{fc > : (■'•. = m}. 
A cycle of the Markov chain (Cj, Mi) is constructed as follows. Define 



(C k ,M k ) 



< k < Gi, 



. (C& - 2h + Cl k _ Gl , GgJ, Gi < k < n, 
with I x = G^-Cg^, n = G x + Tl j. It is not difficult to check that 

[(C k ,M k ),0<k<Ti] 

has the same distribution as the cat and mouse Markov chain on a cycle between two 
meeting times during which an excursion of the cat away from the mouse occurs. 



With the convention to = 0, by induction, denote by tj = t\ 



'j- 



(C k ,M k ) 



21, 



i+1 



ti < k < ti + Gj_|_i, 
ti + Gj + i < k < t i+ i, 



with Ii+i — Cg — _i and t$+i = G,+i + r^i+i- ^ n this way, the sequence 
(C„, M n ) has the same distribution as the Markov chain with transition matrix Q 
defined by Relation (|3|). 

With this representation, the location M n of the mouse at time n is given by 
G? , where 



Kn = X 

i=l 

in particular 
(9) 



^ G* + (n - U) 



4ti_i<n<ti_i+Gi} 



+oo 

E 

1=1 



l/„ IV +1 

£g, < K „ < G £ 



E^ 



6=1 



1 {ti-i+G i <n<ti}; 



with 



Define 



k=l 




L 2,k 



> n 



> n 
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then, for S > 0, on the event {v n > v n + Sy/n}, 

v n +S\/n fn+1 "n+Sy/n v n +l 

n> (2 + Tl k )>J2[Tlk + G k ]+ ]T (2 + Tl k )- £ (G fc - 2) 

fc=l fe=l fe=y„+2 fe=l 

>n + 2 (2 + T|, fe )- ^(G fc -2) 

fe=i/„+2 fc=l 

and, since T\ k < 2 + T% k , the relation 

(io) {]7 n - ^ n > ^v^} c I ^ X! T tk ^ E 



1<£<77„ 

k=l k=l 



holds. For t > 0, define 



(A n (s),0 < s < i) d = ^(^ LniJ - *V sJ ),0 < s < ^ 

for e > 0, by Lemma [5] and the law of large numbers, there exist some xo > and 
no such that if n > no then 

E (G fc -2)>x 0% All <e. 
hence, by Relation (fT0|) one gets that, for n > no and 6 > 0, 



sup A„(s) ><H < 2e + P inf 

0<s<t / \ l<fc<:ro\/n 



fc+L<VnJ \ 

7* ? 



By Lemma [1] the left hand side is thus arbitrarily small if n is sufficiently large. 
In a similar way the same results holds for the variable sup(— A„(s) : < s < t). 
The variable sup(|A n (s)| ; s £) converges therefore in distribution to 0. 
Consequently, by using Relation @ and the law of large numbers, the same property 
holds for 

sup —j= (ny ns \ ~ 2v[ ns \) ■ 

0<s<t V n 

Donsker's Theorem gives that the sequence of processes (C[y^; s j / yfn, < s < t) 
converges in distribution to (Si(s),0 < s < t), in particular, for e and S > 0, there 
exists some no such that if n > no, then 



sup -j= 

,0<u,v<t,\u-v\<6 V n 



> « < e, 



see Billingsley [5] for example. Since M n = for any n > 1, the processes 
t^M l „ sJ , < s < t) and (i<^ Lnsj , < « < * 



have therefore the same asymptotic behavior for the convergence in distribution. 
Since, by construction (C k ) and (V n ) are independent, with Skorohod's representa- 
tion Theorem, on can assume that, on an appropriate probability space with two 
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independent Brownian motions (B\(s)) and (R<2(s)), the convergences 
lim (C"r , / tfi, < a < t) = (Si (a), < s < t), 

lim (77 L , isJ /^) = (W S )/2,0< S <t). 

n— »+oo 

hold almost surely for the norm of the supremum. This concludes the proof of the 
theorem. □ 

3.2. Random Walk in the Plane. The transition matrix P of this random walk 
is given by, for x £ Z 2 , 

p(x,x+(l,0))=p(x,x-(l,0))=p(x,x+(0,l))=p{x,x-(0,l)) - \. 

Definition 1. Let ei = (l,0), e_i=— e\, e2=(0, 1), e_2=— &i o,nd the set of unit 
vectors of 2? is denoted by £ = {ei, e_i, 62, £-2}- 

If (C n ) is a random walk in the plane, (i?„) denotes the sequence in £ such that 
(R n ) is the sequence of unit vectors visited by (C n ) and 

(11) r ef = f - P(iZi = / I Ro = e), eje£. 

A transition matrix Qr on £ 2 is defined as follows, for e, f, g G £. 

{Qli((e,g),(f,g)) =r ef ,e^g, 
Qfl((e,e),(e,-e)) =1/3, 
Qn{{e, e), (e, e)) = Q R ((e, e), (e, -e)) = 1/3, 

with the convention that e, —e are the unit vectors orthogonal to e, /in denotes the 
invariant probability distribution associated to Qr and Dg is the diagonal of £ 2 . 

The transition matrix Qr has some similarity with a cat and mouse Markov 
chain associated to R = (r e f, e, / € £) defined by Equation (fTI]) : as long as the two 
coordinates are not equal, the first one moves according to R. But when they are 
equal, only the second one moves to one of the three other elements with uniform 
probability. 

A characterization of the matrix R. Let 

t + = inf(n > : C n £ £ ) and r = inf(n > : C n £ £), 

then clearly r ef = P(C* T+ = / | C = e). For x £ Z 2 , define 

<f>{x) =P(C T = ei I Co = x), 

by symmetry, it is easily seen that the coefficients of R can be determined by <j>. For 
x £" £ , by looking at the state of the Markov chain at time 1, one gets the relation 

A(f>(x) d = <j>(x+ei) + </>(a;+e_i) + (t>(x+e 2 ) + 0(x+e_ 2 ) - H(x) = 

and 4>(ei) = if i G { — 1, 2, —2} and <f>(ei) = 1. In other words, <j) is the solution of 
a discrete Dirichlet problem: it is an harmonic function (for the discrete Laplacian) 
on I? with fixed values on £ . Classically, there is a unique solution to the Dirichlet 
problem, see Norris |23j for example. An explicit expression of <f> is, apparently, not 
available. 
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Theorem 4. If (Co, Mo) € N 2 , the convergence in distribution of finite marginals 
lim (±M lenti ,t>0) {W{Z{t))l 



holds, with 



,z,/,. = ('B^) Mrt)) . 



where [Ir is the probability distribution on £ 2 introduced in Definition^ the process 
(W(t)) — (Wi(t), W-jft)) * s a dimensional Brownian motion and 

- (Lsit)) the local time at o/ a standard Brownian motion (B(t)) on R 
independent of (W(t)). 

- Fort> 0, T t = mf{s > : B(s) = t}. 

Proof. The proof follows the same lines as before: a convenient construction of 
the process to decouple the time scale of the visits of the cat and the motion of 
the mouse. The arguments which are similar to the ones used in the proof of the 
one-dimensional case are not repeated. 

When the cat and the mouse at the same site, they stay together a geometric 
number of steps whose mean is 4/3. When they are just separated, up to a trans- 
lation, a symmetry or a rotation, if the mouse is at e±, the cat will be at e_2 or 
—e% with probability 1/3. The next time the cat will meet the mouse corresponds 
to one of the instants of visit to £ by the sequence (C n ). If one considers only these 
visits and, up to a translation, it is not difficult to see that the position of the cat 
and of the mouse is a Markov chain with transition matrix Qr. Let (R n ,S n ) be 
the associated Markov chain, for N visits to the set £, the proportion of time the 
cat and the mouse will have met is given by 

JV 



1 



N ^ HRt=S e }, 

this quantity converges almost surely to hr(Ds). 

Now one has to estimate the number of visits of the cat to the set £ . Kasa- 
hara [TB], see also Bingham [6 J and Kasahara [T5], gives that, for the convergence 
in distribution of the finite marginals, the following convergence holds 

f -L B (T t ) 

TT 

The rest of the proof follows the same lines as in the proof of Theorem [3] □ 
Remark. 

Tanaka's Formula, see Rogers and Williams gives the relation 




L(T t ) = t- f * sgn(B(s))dB(s), 
Jo 



10 

where sgn(ir) = — 1 if x < and +1 otherwise. Since the process (T t ) has inde- 
pendent increments and that the T t 's are stopping times, one gets that (L(T t )) 
has also independent increments. The function t — > T t being discontinuous, the 
limiting process (W(Z(t))) is also discontinuous. This is related to the fact that 
the convergence of processes in the theorem is minimal: it is only for the conver- 
gence in distribution of finite marginals. For t > 0, the distribution of L(Tt) is an 
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exponential distribution with mean 2t, see Borodin and Salminen [8] for example. 
The characteristic function of 



Wi 



{16p R (D £ )L(T t 



V 3vr 

at £ € C such that Re(£) = can be easily obtained as 

With a simple inversion, one gets that the density of this random variable is a 
bilateral exponential distribution given by 

a ° f a °\ \\ r- ID 

^expj^Mj^R. 
The characteristic function can be also represented as 

E (^ 1 ™ , ) = ^ = -"(L (^-i)n(M)*), 

with 

g-aoM/ \/t 

n(t, u) = — — : — , u e E, 



u) is in fact the associated Levy measure of the non-homogeneous process 
with independent increments (W\(Z(t))). See Chapter 5 of Gikhman and Sko- 
rokhod [PI. 



4. The Reflected Random Walk 



In this section, the cat follows a simple ergodic random walk on the integers with 
a reflection at 0, an asymptotic analysis of the evolution of the sample paths of the 
mouse is carried out. Despite it is a quite simple example, it exhibits an interesting 
scaling behavior. 

Let P denote the transition matrix of the simple reflected random walk on N, 



(13) 



p(x, x + 1) = p, x > 0, 
p(x, x — 1) = 1 — p, x 7^ 0, 
o(0,0) = l-p. 



It is assumed that p G (0, 1/2) so that the corresponding Markov chain is positive 
recurrent and reversible and its invariant probability distribution is a geometric 

random variable with parameter p d =' p/(l — p). In this case, it is easily checked 
that the measure v on N 2 defined in Theorem [5] is given by 

v{x,y) =p x (l-p), 0<x<y-l, 
v{y-l,y) =py- 1 (l-p)(l-p), 

v(y + l,y) =pv+ 1 (l-p)p, 
v{x,y) =P X (1~P), x>y + l. 
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Proposition 2. //, forn > 1, T n =inf{fc>0 : Ck=n} then, as n goes to infinity, the 
random variable T n /¥,o(T n ) converges in distribution to an exponentially distributed 
random variable with parameter 1 and 

km E (T„)p" = 7^2, 

with p = p/(l — p). 

If Co = n, then Tq/u converges almost surely to (1 — p)/(l + p). 

Proof. The convergence result is standard, see Keilson [17] for closely related re- 
sults. The asymptotic behavior of the sequence (E (T n )) follows simply by getting 
the average hitting time of x starting from x—1 (recurrence relation) and by sum- 
ming up x from 1 to i. □ 

Free Process. Let (C' n , M' n ) be the cat and mouse Markov chain associated to the 
simple random walk on Z without reflection (the free process): 

p'(x, x + l)=p=l— p'(x, x — 1), Vx € 

Proposition 3. If (Cq, Mq) — (0,0) 7 then the asymptotic location of the mouse for 
the free process = lim M' n is such that, for u £ C such that \u\ = 1, 



(14) e(u m ~) 
in particular 



p{\- p)u 2 



-p 2 u 2 + (1 + p)u — 1 ' 



E(M' ao ) = -- p andE^-^j = 1, 



furthermore the relation 

(15) E ( sup jjr ] < +oo 

\n>0 y/P n J 

holds. If (Sk) is the random walk associated to the sequence (Ai) of i.i.d. random 
variables with the same distribution as and (Ei) are i.i.d. exponential random 
variables with parameter (1 + p)/(l — p) 2 , then the random variable W defined by 

(16) W = J2P~ S " Ek > 

k=0 

is almost surely finite with infinite expectation. 

Proof. Let r = inf{n > 1 : C n < M n }, for u € C such that \u\ = 1 then, by looking 
at the different cases, one has 



E 



(u M '^ = (V-p)i+p 2 u^E(V<) +p(l-p)u 



Since, M' T — C' T = 2, after time r, the cat and the mouse meet again with probability 
(p/(l — p)) 2 , consequently, 

l+G 

i=i 

where (M' T A are i.i.d. random variables with the same distribution as M' T and G is 
an independent geometrically distributed random variable with parameter p 2 . This 
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identity gives directly the expression for the characteristic function of and also 
the relation E(M' OD ) = -1/p. 
The upper bound 

supil< < C/ d = 1 + supC; 

n>0 ri>0 

and the fact that U — 1 has the same distribution as the invariant distribution of 
the reflected random walk (C n ), i.e. a geometric distribution with parameter p, give 
directly Inequality (fl~5|) . 

Let N = (N t ) be a Poisson process with rate (1 — p) 2 /(l + p), it is easy to check 
the following identity for the distributions 

r + oo 

(17) W dl = / p- SN <dt, 

Jo 

by the law of large numbers, (Sjvjt) converges almost surely to — (l+p)/[(l — p) 2 p]> 
one gets therefore that W is almost surely finite. From Equation (fT4")l , one gets 
E(u M °°) is well defined for it = 1/p and its value is E(p~ M ° c ) = 1. This gives 
directly that E(W) = +oo. □ 

Note that, as a consequence of this result, the exponential moment E (u M °°^ of 
the random variable is finite for u in the interval [1, 1/p]. 

Exponential Functionals. The representation (|17[) shows that the variable W is 
an exponential functional of a compound Poisson process. See Yor [29 . It can be 
seen as the invariant distribution of the auto-regressive process (X n ) defined as 

X n+1 d = p- A "X n + E ni n>0. 

The distributions of these random variables are investigated in Guillemin et al. |14j 
when (An) are non-negative. See also Bertoin and Yor [4]. The above proposition 
shows that W has a heavy tailed distribution, as it will be seen in the scaling result 
below, this has a qualitative impact on the asymptotic behavior of the location 
of the mouse. See Goldie [13] for an analysis of the asymptotic behavior of tail 
distributions of these random variables. 

A Scaling for the location of the Mouse. The rest of the section is devoted 
to the analysis of the location of the mouse when it is initially far away from the 
location of the cat. Define 

si = m£{e >Q:C e = M e } and t t = inf{£ > si : C e = 0} 
and, for k > 1, 

(18) s fe+ i = inf{£ >t k :d = M e } and t k+1 = M{£ > s k+1 : C e = 0} 

Proposition [2] suggests an exponential time scale for a convenient scaling of the 
location of the mouse. When the mouse is initially at n and the cat at the origin, 
it takes the duration si of the order of p~ n so that the cat reaches this level. 
Just after that time, the two processes behave like the free process on Z analyzed 
above, hence when the cat returns to the origin (at time t-y), the mouse is at position 
n+M^,. The following proposition presents a precise formulation of this description, 
in particular a proof of the corresponding scaling results. For the sake of simplicity, 
and because of the topological intricacies of convergence in distribution, in a first 
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step the convergence result is restricted on the time interval [0, S2], i.e. on the two 
first "cycles". Theorem [5] below gives the full statement of the scaling result. 

Proposition 4. If Mq = n > 1 and Co = then, as n goes to infinity, the random 
variable (Mt 1 —n,p n ti) converges in distribution to (Ai,E\) and the process 



M Lt/> -nj 



{0<t<p"s 2 } 



converges in distribution for the Skorohod topology to the process 



1 



{t<E 1 +p- A iE 2 } 



where A\ is a random variable with the same distribution as defined in Propo- 
sition^ it is independent of E\ and E2, two independent exponential random vari- 
ables with parameter (1 +p)/(l — p) 2 . 

Proof. For T > 0, P([0, T],R) denotes the space of cadlag functions, i.e. of right 
continuous functions with left limits and d° is the metric on this space defined by, 
for x,yeV([0,T\,M), 



d°(x,y) 



inf 

•pen 



sup 

0<s<t<T 



log 



tf(t) - Cfi{s) 



t- s 



■ sup \x((p(s)) - y(s)\ 

0<s<T 



where Tt is the set of non-decreasing functions ip such that (j>(0) = and 4>{T) = T). 
See Billingsley [5]. 

An upper index n is added on the variables s%, 82, t\ to stress the dependence 
on n. Take three independent Markov chains (C£), (C|) and (Cjg) with transition 
matrix P such that Cq = Cq = 0, Cq = n and, for i = a, b, c, denotes 
the hitting time of p > for (CI). Since ((Cfc,Mfc),a" < fc < t") has the same 
distribution as ((n + C^,,M^,),0 < k < T$, by the strong Markov property, the 
sequence (M k , k < s 2 l ) has the same distribution as (N k ,0 < k < T% + T b + T£) 
where 

in, k< T- 

N k = \ n + M k _ Ts , T£<k< T- + T b 

n + M' b , T£ + T b < k < T° + T b + T„ c +M , 
I T b 

where ((Ct,Mi),0 < k < Tq) is a sequence with the same distribution as the 
free process killed at the hitting time of of the first coordinate. Additionally, 
it is independent of the Markov chains (C£) and (C£). In particular, the random 
variable M t .—n has the same distribution as M' b , and since T% converges almost 
surely to infinity, it is converging in distribution . 

Proposition [2] and the independence of (C£) and (Cj£) show that the sequences 
(p n T®) and (p n T^) converge in distribution to two independent exponential ran- 
dom variables E\ and E2 with parameter (1 + p)/(l — p) 2 . By using Skorohod's 
Representation Theorem, see Billingsley [5], up to a change of probability space, it 
can be assumed that these convergences hold for the almost sure convergence. 
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as 



The rescaled process (Mu p -n\ /".l{o<t<p™s 2 }' * — T) has the same distribution 
fl, t<p n \T^, 



/ , >. def. 
X n (t) = < 



1 + -MU, p n \T% + T b l < t < p n \TZ + T b + TZ +M , J, 



n o 

-16 



0, *>PTC+T b + T n c +M ,J, 

for t <T. Proposition[2]shows that Tg/n converges almost surely to (1 — p)/(l + p) 
so that {p n \T% + T b ] ) converges to £i and, for n > 1, 

P T n+M> = P °P ° T n+M' h >P °° E 2, 

almost surely as n goes to infinity. Additionally, one has also 

lim -sup|Affe|=0, 

ra-v+oo n fe > 

almost surely. Define 

X °° = ( 1 {i<TA(Ei+p- M So.E 2 )}) ' 

where a A 6 = min(a, 6) for a, b € K. 

Time change. For n > 1 and t > 0, define w ra [resp. w„] as the minimum [resp. 
maximum] of t A p n \T£ + T b + T£ +M , ] and i A {E x + p- M '- E 2 ), and 

— s, (J < s < u„, 

v n + (s-u n )— , u n <s<T. 

T -u n 

Note that ip n £ 7i, by using this function in the definition of the distance d° on 
T>([0, T], M) to have an upper bound of (d(x n , Xoo)) and with the above convergence 
results, one gets that, almost surely, the sequence (d(x n ,Xoo)) converges to 0. The 
proposition is proved. □ 

Theorem 5 (Scaling for the Location of the Mouse). If Mo = n, Cq — 0, then the 
process 

— ^— W*»> 

converges in distribution for the Skorohod topology to the process (l{ t< yi/}) 7 where 
W is the random variable defined by Equation (|16|) . 
If Hq is the hitting time of by (M n ), 

H a = inf{s > : M s = 0}, 

then, as n goes to infinity, p n Ho converges in distribution to W . 

Proof. In the same way as in the proof of Proposition [H it can be proved that for 
p > 1, the random vector [(M tk — n,p n tk), 1 < k < p] converges in distribution to 
the vector 

/ fc-i \ 



s k ,j2p~ SiE i 



i=0 
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and, for k > 0, the convergence in distribution 

(19) r3+oo ( ^n ^ 1 {°< t <P' 1 ^}) = ( I {t<£ 1 +P- S is 2 +-+P- S ^i£ fc }) 

holds for the Skorohod topology. 

Let 4> : [0, 1] -> M+ be defined by 0(s) = E(p- sAl ), then 0(0) = 4>{1) = 1 and 
</>'(0) < 0, since is strictly convex then for all s < 1, <f>{s) < 1. 

By using Proposition [3] one gets that there exist No, -Ko > and < 5 < 1 such 
that if n > N 

(20) E (n ,„ ) (M il ) E(M tl | M = C =n)<n+ ^E(M^) = n - ^, 
and, by Proposition [21 

(21) P n/2 E(o,n)(V*D <^0, 

and finally, with the identity E(1//9 M °=) = 1, Inequality (fl5]) and Lebesgue's Theo- 
rem, one gets that 

(22) E fp-( M H-»)/ 2 ^ < 5. 

Let ^ = inf{fc > I : M tk < No} and, for k > 1, Gk the <r-field generated by the 
random variables (Cj, Mj) for j < Because of Inequality ([20)) . it is easily checked 
that the sequence 

Mt kA „ + -^(kAv),k>0 
is a supermartingale with respect to the filtration {Gk) hence, 

E (M tkAU ) + 7^E(fc f\v)< E{M ) = n, 
2p 

since the location of the mouse is non-negative, by letting k go to infinity, one gets 
that E(z/) < 2pn. In particular v is almost surely a finite random variable. 

It is claimed that the sequence {p n t u ) converges in distribution to W. For p > 1 
and on the event \y > p}, 

( v -i \ 1/2 

(23) { P n {t v - t p )) 1/2 = J2 - **) < E y/p n itk+i-t k ). 

\k=p J k=p 

For k > p, Inequality ([21]) and the strong Markov property give the relation 

/W 2 E [yt fc+ i - i fc | &] = p M '*/ 2 E (0>Mtfc) [Vti] < #o 
holds on the event -{V > k} C {M tfc > iVo}. One gets therefore that 
E(Vp"(*fc+i -**)!{*<,}) -E(p ( "-^ )/2 l {fe< , } p M ^ /2 E [^fc+i-tfc I &]) 

<# E (p(- M '^/ 2 l {fc <, } ) 
holds and, with Inequality (|22p and again the strong Markov property, 
E (p(- A ^/ 2 l {fe< , } ) = E (p-^^i+i-^V'l^j) 



< 



JE ^-^o 2 (^ +1 -M t ,)/ 2l{fc _ 1<i/} )<^ 
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Relation (j23|) gives therefore that 



E 



y/p n (t»-t P ) 



< 



K SP 
1-5' 



For £ > 0, 



E 



1 - e 



+ P(v <p) 



-\-oo 



(24) 



< 



£e-t y V(p n (t v - tp) >u)du + P(i/ < p) 



[ + °° Z-^du + P(v<p), 
1 — Jo V u 



by using Markov's Inequality. Since p n t p converges in distribution to E + p~ Sl E\ + 
■ • ■ + p~ Sp E p , one can prove that, for e > 0, by choosing a fixed p sufficiently large 
and that if n is large enough then the Laplace transforms at £ > of the random 
variables p n t v and W are at a distance less than e. 

At time t v the location M tv of the mouse is x < Nq and the cat is at 0. Since the 
sites visited by M n is a Markov chain with transition matrix {p{x,y)), with prob- 
ability 1, the number R of jumps for the mouse to reach is finite. By recurrence 
of (C„), almost surely, the cat will meet the mouse R times in a finite time. Con- 
sequently, by the strong Markov property, the difference Hq — t v is almost surely a 
finite random variable, the convergence in distribution of (p n Ho) to W is therefore 
proved. □ 

Non-convergence of scaled process after W. Theorem [5] could suggest that 
the convergence holds for a whole time axis, i.e., 



lim (^fci )t > ]=|l 



{t<w} 



t>0), 



for the Skorohod topology. That is, after time W the rescaled process stays at 
like for fluid limits of stable stochastic systems. The next proposition shows that 
this convergence does not hold at all. 



Proposition 5. If Mo— Co— then for any s, t > with s < t, the relation 

Mi 

2, 

holds. 



It should be kept in mind that, since (C„, M n ) is recurrent, the process (M n ) returns 
infinitely often to so that Relation ([25]) implies that the scaled process exhibit 
oscillations for the norm of the supremum on compact intervals. 

Proof. First it is assumed that s = 0. If Co = and T = infjfc > : Ck = 0}, 
then in particular E(T ) = 1/(1 — p). The set C = {Co, . . . , Ct -i} is a cycle 
of the Markov chain, denote by B its maximal value. The Markov chain can be 
decomposed into independent cycles (C n ,n > 1) with the corresponding values 
(Tq) and (B n ) for To and B. Kingman's result, see Theorem 3.7 of Robert [25] for 
example, shows that there exists some constant Kq such that F(B > n) ~ Kop n . 



22 



N. LITVAK AND PH. ROBERT 



Take < 5 < 1/2, for a > 0, 

U n d ^ p^ n 1 £ J [l {Bk > Sn} - F(B > Sn)} , 

k=l 

then, by Chebishov's Inequality, for e > 0, 

H\Un\ >e)< ^p^- 2S ^F(B > Sn) < ^-p^ n , 

for some constant K. By using Borel-Cantelli's Lemma, one gets that the sequence 
(U n ) converges almost surely to 0, hence almost surely 

(26) lim p^ n V l {Bk>Sn} = aK . 

k=\ 

For x G N, let v x be the number of cycles up to time x, the strong law of large 
numbers gives that, almost surely, 



1 x 

v x 1 x ^ 

lim — = lim - > l{c fe =o} = 1 - P- 



a:— >+oo X a:— >+oo X 

i=l 



Denote by x n = [p n t\ , for ao > 0, the probability that the location of the mouse 
is never above level Sn on the time interval (0, x n ] is 

(27) P I sup M fe < Sn I < 

\i<fe<L,o-"tJ / 

Pf sup M k <Sn,p^\\ {Bt>5n} >^) 



(l-i5)n 

If ao is taken to be (1 — p)t and if a\ = aoKo/2, by Equation ([26]) . one gets that 
the last expression converges to as n gets large. Since the sequence of successive 
sites visited by the mouse is a also a simple reflected random walk, 

sup M k < Sn,p^ n V l {Bi > 4n} > ai I 
i<fe<Lp-"tj , =0 / 

/ lp- n t\ 

< P sup M fe < Sn,p^ n V l {Ci=Mi} > ai 
yi<fc<L P -™tJ i=0 

< P ( sup C fe < 5n ) = P (r L5nJ+1 > La ip - (1 - 5)n j) 

\l</c<Laip-< 1 -' 5 )"J / V 7 

with the notations of Proposition ® but this proposition shows that the random 
variable pL 5 "J Ti 5n j +1 converges in distribution as n gets large. Consequently, since 
S < 1/2, the expression 
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converges to 0. The relation 

Um P f sup ifcj >1\ =1 

n->+oo \o<„<t n 2J 

has been proved. 

The proof of the same result on the interval [s,t] uses a coupling argument. 
Define the cat and Mouse Markov chain (C^, Mk) as follows: 

(Cfc,fc>0) = (C L . p -» J+fc ,fc>0) 

and the respective jumps of the sequences (-Mup-nj+fc) and (M k ) are independent 
except when My sp - n ^ +k = M k in which case they are the same. In this way, one 
checks that (C k ,M k ) is a cat and mouse Markov chain with the initial condition 

(Co,M ) = (C L . p -»j,0). 

By induction on k, one gets that Mj sp -„j +fe > M k for all k > 0. Because of the 
ergodicity of (C k ), the variable C\ sp -n\ converges in distribution as n get large, in 
the same way as before, one gets that 

lim p I sup ^"P-J >i)=l, 
n ^+°° \o<u<t- s n 2 J 

therefore 

liminfpf sup Ml " p '" j > - ) > liminf P f sup M L«P~ ra J > I ] = L 
n^+oo \ s < u < t n 2/ n^+oo \^o<«<t-s n 2y 

This completes the proof of Relation ([25]) . 

□ 

It is very likely that the relation 

Um P f sup A W-J = n = ! 
n^+oo \ s < u < t n 2/ 

holds in fact for the following intuitive (and non-rigorous) reason. Each time the 
cat meets the mouse at x large, the location of the mouse is at x + when 
the cat returns to 0, where M'^ is the random variable defined in Proposition [3] 
In this way, after the nth visit of the cat, the mouse is at the nth position of a 
random walk associated to M'^ starting at x. Since E(l/p M °°) = 1, Kingman's 
result, see Kingman |19j . implies that the hitting time of 5n by this random walk 
is of the order of p~ 5n . For each of the steps of the random walk, the cat needs 
also of the order of p~ Sn units of time. Hence to reach level 5n, p~ 2Sn units of 
time are required, this happens on the time scale t — > p~ n t only if 5 < 1/2. The 
difficulty is that the mouse is not at x + M'^ when the cat returns at at time t x 
but at x + M' T , so that the associated random walk is not space-homogeneous but 
only asymptotically close to the one described above. Since an exponentially large 
number of steps of the random walks are considered, controlling the accuracy of 
the approximation turns out to be a problem. 
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5. Continuous Time Markov Chains 

Let Q — (q(x,y),x,y € S) be the Q-matrix of a continuous time Markov chain 
on iS such that, for any i£(S, 

qx d = q ( x > y ^ 

is finite and that the Markov chain is positive recurrent and 7r is its invariant 
probability distribution. The transition matrix of the underlying discrete time 
Markov chain is denoted as p(x, y) = q(x, y)/q x , for x ^ y, note that p{-, •) vanishes 
on the diagonal. For an introduction on Markov chains see Norris [23] and Rogers 
and Williams [27| for a more advanced presentation. 

The analogue of the Markov chain (C„, M n ) in this setting is the Markov chain 
(C(t),M(t)) on >S 2 whose infinitesimal generator fl is defined by, for x, y G S, 

(28) ft(f)(x, y) = Y, l( x > V) ~ y)]M^y} 

+ ^2 qxP(x,z)p(x,z')[f(z,z') - f(x,x)]l {x=y} 

for any function / on S 2 vanishing outside a finite set. The first coordinate is 
indeed a Markov chain with Q-matrix Q and when the cat and the mouse are at 
the same site x, after an exponential random time with parameter q x they jump 
independently according to the transition matrix P. Note that if one looks at the 
sequence of sites visited by (C(i), M{t)) then it has the same distribution as the cat 
and mouse Markov chain associated to the matrix P. For this reason, the results 
obtained in Section[2]can be proved easily in this setting. In particular (C(<), M(t)) 
is null recurrent when (C(i)) is reversible. 

Proposition 6. //, for t > 0, 

U{t) = I l{jW( s )=C(s)} ds 

Jo 

and S(t) — inf{s > : U(s) > t} then the process (M(S(t))) has the same distri- 
bution as (C(t)), i.e. it is a Markov process with Q-matrix Q. 

This proposition simply states that, up to a time change, the mouse moves like 
the cat. In discrete time this is fairly obvious, the proof is in this case a little more 
technical. 

Proof. If / is a function S, then by characterization of Markov processes, one has 
that the process 

(H(t)) d ^ (f(M(t)) - /(M(0)) - J* Q(f)(C(s), M(s)) ds^j 

is a local martingale with respect to the natural filtration {T t ) of (C(t), M (t)), 
where / : S 2 — * R such that f(x,y) — f(y) for x, y € S. The fact that, for 
t > 0, S(t) is a stopping time, that s —> S(s) is non-decreasing and Doob's optional 
stopping theorem imply that (H(S(t) j) is a local martingale with respect to the 
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filtration (JFs(t))- Since 

S{t) Q(f)(C(s), M(s)) ds = V f S(t) q(M( S ),y)l {c{s)=M(s)} (f(y) - f(M(s))) ds 

yes Jo 
S(t) 

l{C(s)=M(s)}Q(f){M{s))ds 

Q(f)(M(S(s)))ds, 



one gets therefore that 

(7(M(S(t))) - /(M(Q)) - J Q(f)(M(S(s))) ds 

is a local martingale for any function / on S. This implies that (M(S(t))) is a 
Markov process with Q-matrix Q, i.e. that (M(S(t))) has the same distribution as 
(C(i)). See Rogers and Williams [26]. □ 

The example of the M/M/oo process. The example of the M/M/oo queue 
is investigated in the rest of this section. The associated Markov process can be 
seen as an example of a discrete Ornstein-Uhlenbeck process. As it will be shown, 
there is a significant qualitative difference with the example of Section [4] which is a 
discrete time version of the M/M/l queue. The Q-matrix is given by 



(29) 




P, 

x 



The corresponding Markov chain is positive recurrent and reversible and its invari- 
ant probability distribution is Poisson with parameter p. 

Proposition 7. If C(0) = x < n — 1 and 

T n = inf{s > : C(s) = n}, 

then, as n tends to infinity, the variable T n / E x (T n ) converges in distribution to an 
exponentially distributed random variable with parameter 1 and 

lim E x {T n )p n /{n-l)\ = e-P. 

n — >+oo 

J/C(0) = n, then Tq/ log n converges in distribution to 1. 

See Chapter 6 of Robert [5S]. It should be remarked that the duration of time it 
takes to reach n starting from is essentially the time it takes to go to n starting 
from n — 1 . 

Proposition 8. J/C(0) = M(0) = n, and 

T a = inf{s > : C(s) = 0}, 

then, as n goes to infinity, the random variable M(Tq)/u converges in distribution 
to a random variable F on [0, 1] such that P(F < x) — x p . 

Proof. Let r = inf{s > : M(s) — M(s— ) + 1} be the instant of the first upward 
jump of (M(s)). Since (M(S(s))) has the same distribution as (C(s)) one gets that 
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U(t) is an exponential random variable with parameter p and, since there is no 
arrival up to time r, then 

n 

M(r) d ^l + ^l {£i>t/(r)} , 

i=l 

where (Ei) are i.i.d. exponential random variables with parameter 1. For 1 < i < n, 
Ei is the service time of the ith initial customer. At time r, the process of the 
mouse will have run only for U(t), so the ith customer is still there if Ei > U(t). 
Consequently, by conditioning on the value of U(t), by the law of large numbers 
one obtains that the sequence (M(r)/n) converges in distribution to the random 

variable F d = exp(— U(t)). 

The dynamic of the cat and mouse gives that: 

- On the event t > Tq, necessarily M(t— ) = C(r— ) = 0, thus the quantity 
P(t > T ) < P(M(t) = 1) converges to 0. 

- Just before time r, the mouse and the cat are at the same location and 

M(r-) 



P(C(t) = M(t-) - 1) = E 



P + M(t-) 



converges to 1 as n gets large. 
If e > 0, then 

1 (Pm(t)-i(To > T M{T) )) < P (MM <e) + sup P fe (T > T k+1 ), 

\ n J k>len\ 

hence, by Proposition [3 for e [resp. n] sufficiently small [resp. large], the above 
quantity is arbitrarily small. This result implies that the probability of the event 
{M(t) = M(Tq)} converges to 1. The Proposition is proved. □ 

A Multiplicative Phenomenon. If C(0) = and M(0) = n, the next time the 
cat returns to 0, Proposition [8] shows that the mouse will be at a location of the 
order of nF\, where Fi = exp(—Ei/p) and E\ is an exponential random variable 
with parameter 1 . After the pth round, the location of the mouse is of the order of 

v 

(30) nJ^F/ £ =nexp 

fc=i 

where {Eh) are i.i.d. with the same distribution as E±. A precise statement of this 
non-rigorous statement can be formulated easily. From Equation (|30p . one gets 
that after a Poisson number of rounds with parameter p log n, the location of the 
mouse is within a finite interval. 

The corresponding result for the reflected random walk exhibits an additive 
behavior. Theorem [5] gives that the location of the mouse is of the order of 

v 

(31) n + ^Ai 

i=l 

after p rounds, where the common distribution of the (Ak) is given by the generating 
function of Relation (fl4]) . In this case the number of rounds after which the location 
of the mouse is located within a finite interval is of the order of n. 

As Theorem [5] shows, for the reflected random walk, t — > p~ n t is a convenient 
time scaling to describe the location of the mouse until it reaches a finite interval. 
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This is not the case for the M/M/oo queue, since the duration of the first round of 
the cat, of the order of (n — by Proposition [3 dominates by far the duration 

of the subsequent rounds, i.e. when the location of the mouse is at xn with x < 1. 
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